Deep-learning-based technologies such as deepfakes ones have been attracting widespread attention in both society and academia, particularly ones used to synthesize forged face images. These automatic and professional-skill-free face manipulation technologies can be used to replace the face in an original image or video with any target object while maintaining the expression and demeanor. Since human faces are closely related to identity characteristics, maliciously disseminated identity manipulated videos could trigger a crisis of public trust in the media and could even have serious political, social, and legal implications. To effectively detect manipulated videos, we focus on the position offset in the face blending process, resulting from the forced affine transformation of the normalized forged face. We introduce a method for detecting manipulated videos that is based on the trajectory of the facial region displacement. Specifically, we develop a virtual-anchor-based method for extracting the facial trajectory, which can robustly represent displacement information. This information was used to construct a network for exposing multidimensional artifacts in the trajectory sequences of manipulated videos that is based on dual-stream spatial-temporal graph attention and a gated recurrent unit backbone. Testing of our method on various manipulation datasets demonstrated that its accuracy and generalization ability is competitive with that of the leading detection methods.

COVID-19大流行已经暴露了全球医疗服务的脆弱性，增加了开发新颖的工具来提供快速且具有成本效益的筛查和诊断的需求。临床报告表明，Covid-19感染可能导致心脏损伤，心电图（ECG）可以作为Covid-19的诊断生物标志物。这项研究旨在利用ECG信号自动检测COVID-19。我们提出了一种从ECG纸记录中提取ECG信号的新方法，然后将其送入一维卷积神经网络（1D-CNN）中，以学习和诊断疾病。为了评估数字信号的质量，标记了基于纸张的ECG图像中的R峰。之后，将从每个图像计算的RR间隔与相应数字化信号的RR间隔进行比较。 COVID-19 ECG图像数据集上的实验表明，提出的数字化方法能够正确捕获原始信号，平均绝对误差为28.11 ms。我们提出的1D-CNN模型在数字化的心电图信号上进行了训练，允许准确识别患有COVID-19和其他受试者的个体，分类精度为98.42％，95.63％和98.50％，用于分类COVID-19 vs.正常，与正常人分类， COVID-19与异常心跳和Covid-19和其他类别分别与其他阶级。此外，提出的方法还为多分类任务实现了高级的性能。我们的发现表明，经过数字化的心电图信号训练的深度学习系统可以作为诊断Covid-19的潜在工具。

我们已经调查了对抗性示例的新应用，即对地标识别系统的位置隐私保护。我们介绍了掩模引导的多模式投影梯度下降（MM-PGD），其中对抗示例在不同的深层模型上进行了培训。图像内容受到分析区域的特性，以识别最适合在对抗示例中混合的区域的性质。我们研究了两种区域识别策略：基于类激活图的MM-PGD，其中训练有素的深层模型的内部行为是针对的；和基于人视觉的MM-PGD，其中吸引人类注意力较少的地区的目标是针对的。Ploce365数据集的实验表明，这些策略在不需要大量图像操作的情况下可能有效地防御Black-Box Landmark识别系统。

深神经网络容易受到对抗的例子（AES）的伤害，这具有对抗性转移性：为源模型产生的AES可以误导另一个（目标）模型的预测。然而，从阶级目标模型的预测被误导的角度来看，尚未理解的可转换性尚未理解（即，传播的可传送性）。在本文中，我们将目标模型预测与源模型（“相同错误”）或不同的错误类（“不同错误”）进行分析，以分析和提供对机制的解释。首先，我们的分析显示（1）与“非目标转移性”和（2）不同的错误在类似模型之间发生不同的错误，而不管扰动大小如何。其次，我们提出了一种证据表明，相同的差异和不同的错误可以通过非稳健的特征来解释，预测性但人的无法解释的模式：当AES中的非鲁棒特征被模型使用时发生不同的错误。因此，非鲁棒特征可以为AES的类感知转换性提供一致的解释。

估计公共场所的面膜磨损比率很重要，因为它使卫生当局能够及时分析和实施政策。报道了基于图像分析估计掩模磨损比的方法。但是，仍然对两种方法和数据集仍然缺乏全面的研究。最近的报告通过应用常规物体检测和分类方法直接提出估算比例。使用基于回归的方法来估计佩戴面具的人数是可行的，特别是对于具有微小和遮挡面孔的拥挤场景，但这并未得到很好的研究。大规模和良好的注释数据集仍在需求。在本文中，我们提出了两种比率估计方法，其利用基于检测的或基于回归的方法。对于基于检测的方法，我们改进了最先进的面部探测器，RetinaFace，用于估计比率。对于基于回归的方法，我们微调基线网络CSRNet，用于估计屏蔽和未屏蔽面的密度图。我们还提供了第一个大规模数据集，其中包含从18,088个视频帧中提取的581,108脸注释，从17个街道视图视频中提取了581,108个脸部注释。实验表明，基于视网膜的方法在各种情况下具有更高的准确性，并且由于其紧凑性，基于CSRNet的方法具有更短的操作时间。

Diabetic Retinopathy (DR) is a leading cause of vision loss in the world, and early DR detection is necessary to prevent vision loss and support an appropriate treatment. In this work, we leverage interactive machine learning and introduce a joint learning framework, termed DRG-Net, to effectively learn both disease grading and multi-lesion segmentation. Our DRG-Net consists of two modules: (i) DRG-AI-System to classify DR Grading, localize lesion areas, and provide visual explanations; (ii) DRG-Expert-Interaction to receive feedback from user-expert and improve the DRG-AI-System. To deal with sparse data, we utilize transfer learning mechanisms to extract invariant feature representations by using Wasserstein distance and adversarial learning-based entropy minimization. Besides, we propose a novel attention strategy at both low- and high-level features to automatically select the most significant lesion information and provide explainable properties. In terms of human interaction, we further develop DRG-Net as a tool that enables expert users to correct the system's predictions, which may then be used to update the system as a whole. Moreover, thanks to the attention mechanism and loss functions constraint between lesion features and classification features, our approach can be robust given a certain level of noise in the feedback of users. We have benchmarked DRG-Net on the two largest DR datasets, i.e., IDRID and FGADR, and compared it to various state-of-the-art deep learning networks. In addition to outperforming other SOTA approaches, DRG-Net is effectively updated using user feedback, even in a weakly-supervised manner.

Research has shown that climate change creates warmer temperatures and drier conditions, leading to longer wildfire seasons and increased wildfire risks in the United States. These factors have in turn led to increases in the frequency, extent, and severity of wildfires in recent years. Given the danger posed by wildland fires to people, property, wildlife, and the environment, there is an urgency to provide tools for effective wildfire management. Early detection of wildfires is essential to minimizing potentially catastrophic destruction. In this paper, we present our work on integrating multiple data sources in SmokeyNet, a deep learning model using spatio-temporal information to detect smoke from wildland fires. Camera image data is integrated with weather sensor measurements and processed by SmokeyNet to create a multimodal wildland fire smoke detection system. We present our results comparing performance in terms of both accuracy and time-to-detection for multimodal data vs. a single data source. With a time-to-detection of only a few minutes, SmokeyNet can serve as an automated early notification system, providing a useful tool in the fight against destructive wildfires.

In the era of Internet of Things (IoT), network-wide anomaly detection is a crucial part of monitoring IoT networks due to the inherent security vulnerabilities of most IoT devices. Principal Components Analysis (PCA) has been proposed to separate network traffics into two disjoint subspaces corresponding to normal and malicious behaviors for anomaly detection. However, the privacy concerns and limitations of devices' computing resources compromise the practical effectiveness of PCA. We propose a federated PCA-based Grassmannian optimization framework that coordinates IoT devices to aggregate a joint profile of normal network behaviors for anomaly detection. First, we introduce a privacy-preserving federated PCA framework to simultaneously capture the profile of various IoT devices' traffic. Then, we investigate the alternating direction method of multipliers gradient-based learning on the Grassmann manifold to guarantee fast training and the absence of detecting latency using limited computational resources. Empirical results on the NSL-KDD dataset demonstrate that our method outperforms baseline approaches. Finally, we show that the Grassmann manifold algorithm is highly adapted for IoT anomaly detection, which permits drastically reducing the analysis time of the system. To the best of our knowledge, this is the first federated PCA algorithm for anomaly detection meeting the requirements of IoT networks.

Collecting large-scale medical datasets with fully annotated samples for training of deep networks is prohibitively expensive, especially for 3D volume data. Recent breakthroughs in self-supervised learning (SSL) offer the ability to overcome the lack of labeled training samples by learning feature representations from unlabeled data. However, most current SSL techniques in the medical field have been designed for either 2D images or 3D volumes. In practice, this restricts the capability to fully leverage unlabeled data from numerous sources, which may include both 2D and 3D data. Additionally, the use of these pre-trained networks is constrained to downstream tasks with compatible data dimensions. In this paper, we propose a novel framework for unsupervised joint learning on 2D and 3D data modalities. Given a set of 2D images or 2D slices extracted from 3D volumes, we construct an SSL task based on a 2D contrastive clustering problem for distinct classes. The 3D volumes are exploited by computing vectored embedding at each slice and then assembling a holistic feature through deformable self-attention mechanisms in Transformer, allowing incorporating long-range dependencies between slices inside 3D volumes. These holistic features are further utilized to define a novel 3D clustering agreement-based SSL task and masking embedding prediction inspired by pre-trained language models. Experiments on downstream tasks, such as 3D brain segmentation, lung nodule detection, 3D heart structures segmentation, and abnormal chest X-ray detection, demonstrate the effectiveness of our joint 2D and 3D SSL approach. We improve plain 2D Deep-ClusterV2 and SwAV by a significant margin and also surpass various modern 2D and 3D SSL approaches.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.